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ABSTRACT 

An iterative least squares procedure for analyzing 
the effect of varions kinds of intervention in time*ser£es data is 
describee^ ^ There are numerous applications of this design in 
economics, education, and psychology, although until recently, no 
appropriate analysis techniques had been developed to deal with the 
model adequately. This paper presents afid develops a complex example 
of time-series experiments using simulated data with the intent of 
illustrating the analytic power of the technique for educational 
methodologists in a number of substantive fields. The simulated data 
were developed to conform to an autoregressive integrated moving 
averages (ASIMA) model, and three intervention effects were built 
into t^e series. The first intervention exerted a constant effect; 
the second damped the effect, and the third caused a general drift 
(trend shift) of the data points. The methodology and results of the 
simulation— including the simulation data — are provided. 
(Author/DGC) 
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Introduction 

An iterative least squares procedure for analyzing the effect of various 

i 

kinds of intervention in time-series data was originally developed by Box " 
and Tiao (1965) and has recently been extended by Glass, Wlllson, and Gottman 
(1974). This procedure provides a method of analysis appropriate for the 
time-series quasi-experimental design of Campbell and Stanley (1963) where 
the data consist of a series of observations with one or more interventions 
(treatment) in the series. There are numerous applications of this design 
in economics, education, and psychology, although until recently, no appro- 
priate analysis techniques had been developed to deal with the model 
adequately. 

This paper presents and develops a complex example of a time-series experiment 
using simulated data with the intent of illustrating the analytic power of a 
time-series experimental analysis for educational methodologists in a variety 
of substantive fields. 

The simulated data were developed to conform to an autoregressi ve integrated 
moving averages (ARIMA) model of order (2,1,1); i.e., the autoregressi ve 
process was of second order, and the differencing and moving averages processes 
were of first order. Three intervention effects were built into the 200 
observations after the 50th, 100th, and 150th data points. The first 
intervention exerted a constant effect on the series; the second, a damped 
effect, and the third caused a general drift (trend shift) of the data points. 
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While it is true that in practice one would rarely encounter three such 
divergent intervention effects on the same series of data, these three 
effects were selected for illustration, and do, in fact, represent 
commonly encountered intervention or treatment effects in real data. A 
constant intervention effect can be illustrated by the hypothetical situation 
where in an all-male college, the level (mean) of SAT scores for freshmen 
increases by a constant amount (measured over a number of years) due to the 
intervention of allowing the university to become co-educational. An inter- 
vention that exerts a damped effect on the initial level of a series is 
illustrated by the well-known phenomenon of the Hawthorne effect in behavioral 
research. In this situation, the scores on tests in a particular classroom 
might initially rise after introduction of a new classroom approach (treatment 
or intervention), but would gradually decline to the original level over time 
as the effect of the intervention or the novelty of the situation wore off. 
The third type of intervention illustrated in this paper, that of a trend or 
drift effect, is found in practical situations in which the treatment not only 
changes the initial mean of the series, but continues to exert an effect by 
changing the slope of the observations. For example, the distance a student 
in a physical education class can run might be constant in the first several 
weeks of the class. Introduction of a required period of calisthenics preceding 
each attempt to run .might enable the student to increase his running speed by 
a few seconds every week for the remainder of the semester. 
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Method 

A computer program was developed to generate data conforming to the ARIMA 
(2,1,1) model described above. The two autoregressive parameters were set at 
-.3 and .6, respectively, and the moving averages parameter at .5; the initial 
level of the series was set to zero and the error variance to unity. The 
three interventions, after the 50th, 100th, and 150th observations, consisted 
of: 

(1) a constant effect of 10 points; i.e., 
z* = + 10 t = 51 to 100 



(2) a damped effect; i.e.. 



^ 2K+99 " ^2K+99 * 



2K-1 



^2K+100 " ^2K+100 * 



2K-1 



> K = 1 , 25 



(3) a trend effect; i.e., 
* t 

z. = 2^. + z (.2) t = 151 to 200 
^ ^ i=l 

Autocorrelations and partial autocorrelations were then computed for each set 
of 50 observations separately. Table 1 shows these results. 
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Identification of the proper model from the pattern of autocorrelation and 
partial autocorrelation coefficients becomes increasingly difficult as the 
true model increases in complexity. In the present example, the model is 
quite complex and consequently exact model identification is difficult. 
Fortunately, the>^e are frequently two or more models of the ARIMA (p,d,q) 
form that will satisfactorily fit the data. To illustrate this point, three 
different models were identified from the pattern of autocorrelation and 
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partial autocorrelation coefficients. These were: 

(1) The true ARIMA model (2,1,1) where, 

£^ - L+a^ - e^a^^^ + (^^ + l)(2^^^ - l) + (^^ - ^i) (2^.2 - L) 

(2) The ARIMA model (2,0,1) where, 

= L + a^ - QiVl *i(2t.l - ♦2(2t.2 " 

(3) The ARIMA model (1,1,1) where, 

= L + -QiVl ^ 1) (Vl - ^ ^2(^2 - ^) 

In each model, £^ is the observation at time t^ 

^t is the error component at time t^ 
L is the initial level 

61 is the first order moving averages parameter 
is the first order autoregressive parameter 
(j)2 is the second order autoregressive parameter 

The simulated data were then run through a computer program for analyzing 
time-series experir-ents to compute least squares estimates of the initial 
level of the series and of the three intervention effects* Three such 
computer runs were made to obtain the parameter estimates for each of the 
potential models identified *from the autocorrelation and partial autocorrelation 
coefficients* In each case, a design matrix was constructed (as input for the 
program) which specified the form of the parameter effects to be estimated. (The 
design matrix is shown in the appendix.) 
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The final step in this project was to assess the goodness of fit of each of 
the least squares solutions. To accomplish this, the residuals after fitting 
the model were computed for each observation. These residual data were then 

2 

auto correlated, and a x statistic computed which tested the independence 

/ 2 

of the residuals. (The x statistic is a function of the sum of squared 
autocorrel ati on coef f i cients • ) 

Results and Discussion 

Table Z summarizes the results of analyzing the data as time-series experiments 
under three different ARIMA models. All three produced parameter estimates 
of initial level and intervention effects near the true values built into 
the data with the exception of the (2,0,1) model estimate of initial level. 
The true initial level of the series was zero, and the (1,1,1) and (2,1,1) 
models yielded estimates of .41 and .50, respectively, both not significantly 
different from zero* The (2,0,1) model, however, estimated the initial level 
of the series as 2.61 which was significantly different from zero with a t^ 
statistic value of 3.96. Undoubtedly, this estimate was large because the 
original data required differencing in order to achieve stationarity, yet the 
(2,0,1) model specifies that no differencing is necessary. 

The estimated values of the first intervention effect were 9.74, 10.01, and 
9.87, respectively, for the (2,1,1), 2,0,1), and (1,1,1) models, all of 
which were highly significantly different from zero and near the true value 
of 10-(l. Similarly, the estimates for the second intervention effect v/ere 
all near 10.0 and highly significantly different from zero. The three models 
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produced estimates of the third intervention parameter of .31, .46, and .30, 
respectively, for the (2,1,1), (2,0,1), and (1,1,1) model, all of v/hich were 
reasonably close to the true value of 0.2. 

The error variance estimates were also near the true value of unity in all 
three cases, the smallest value yielded by the (2,1,1) model, the middle 
value by the (1,1,1), and the largest value by the (2,0,1) model. 

2 

The X test statistics of the residuals after fitting each model were all 
non-significant indicating that any of the three models was adequate for 

2 

describing the data according to the x criterion; the (2,1,1) model had the 

2 

loV/est X value, however. 

The estimates of (j)2, and 0i for each model were, of course, quite 
different across the three models. Probably the most interesting of these, 
is the result for the (2,1,1) model, since the true autoregre^sive and moving 
averages parameters were specified in the construction of the model, and can 
be compared to the estimated values. The values were originally specified 
as (jii = -.3, (j)2 = .6, and Oi = .5. The minimum error variance was found 
at = -.6, (j)2 = .3, and 62 = .1. The differences between the two sets of 
values prohably resulted fror. sairpling error during generation of the data 
(using the random number generator to build random error into the model). 
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Summary 

Simulated data were generated to conform to an ARIMA (2,1,1) process, and 
three intervention effects were built into the series. Analysis of the series 
was performed in two stages. First, separate correlograms of the data 
were inspected before and after each intervention to verify that the time- 
series conformed to an ARIMA model of order (2,1,1). 

Second, the data were analyzed using a program specifically developed for 
analyzing time-series data with one or more interventions. A design matrix 
was specified for the four parameters to be estimated (i.e., the initial 
level of the series, and th^ three intervention effects), incorporating a 
constant, a damped, and a change in drift intervention effect. Values of 
the two autoregressive parameters, <(>i, and 91^ and the moving averages 
parameter, e^, were examined over the invertibility and stationarity regions 
to determine the minimum residual error term, and thus, the best fit for the 
model, using the least squares criterion. The values of ^ and 0 which 
minimized the residual error variance were found to be similar to and within 
an acceptable range of the values used to build the simulated data. Examination 
of the corresponding parameter estimates and associated t^ statistics for the 
initial level of the series and the three intervention effects also revealed 
nearly exact estimates of the true parameter values which were highly 
significant for all design effects. 
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